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ABSTRACT 

The  simplest  target  selection  problem  in  heterogeneous 
combat  is  the  two-on-one  battle.   For  a  prescribed  duration 
battle,  deterministic  and  stochastic  models  using 
Lanchester's  square  law  attrition  mechanism  are  developed. 
Solutions  of  these  models,  obtained  by  the  application  of 
optimal  control  theory,  are  given,  including  the  complete 
solution  of  the  deterministic  model  and  the  optimal 
target  selection  rule  and  expected  pay-off  for  special 
cases  of  the  stochastic  model.   A  numerical  approximation 
for  the  general  case  of  the  stochastic  model  is  obtained. 
Comparison  of  numerical  results  shows  that  the  target 
selection  rule  specified  by  the  deterministic  models 
differs  from  the  one  given  by  the  stochastic  model. 
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I.   INTRODUCTION 

One  of  the  chief  functions  of  mathematical  models 
of  combat  is  to  provide  insight  for  the  development  of 
tactics.   In  general,  the  assumptions  needed  to  make  the 
model  tractable  are  such  that  the  results  cannot  be  direct- 
ly applied.   Frequently,  however,  the  results  reveal  a 
structure  which  is  obscured  by  the  complexity  of  the  real 
world  problem. 

There  are  a  number  of  different  approaches  to  combat 
attrition  modeling,  each  with  its  own  advantages  and  dis- 
advantages.  Generally  speaking,  greater  reality  is  intro- 
duced into  such  models  at  the  expense  of  increased 
complexity . 

A  factor  to  be  considered  is  whether  the  results  are 
influenced  by  the  approach  used.   It  may  be  that  for  a 
particular  type  of  problem,  a  complex,  difficult  approach 
yields  nothing  that  could  not  be  gained  from  a  simpler 
model . 

One  of  the  first  mathematical  models  of  combat  was 
presented  by  Rear  Admiral  Bradley  A.  Fiske,  USN,  in 
1905  [1].   Fiske  considered  an  engagement  between  two 
fleets.   He  assigned  a  numerical  value  representing  strength 
to  each  fleet  and  assumed  that  the  damage  a  fleet  could 
inflict  was  proportional  to  its  strength.   In  a  series 
of  tables,  he  showed  that  the  value  of  the  weaker  fleet 


in  relation  to  the  stronger  fleet  decreased  at  an  acceler- 
ating rate  as  the  battle  progressed.   Fiske's  logical 
development  and  conclusions  are  the  same  as  those  later 
formalized  by  Lanchester. 

Pioneering  work  on  combat  attrition  modelling  was  done 
by  F.  W.  Lanchester  [2]  in  the  early  nineteen  hundreds. 
He  postulated  that  the  combat  attrition  process  could  be 
described  by  a  system  of  ordinary  differential  equations. 

Lanchester  considered  two  types  of  combat:  individual 
combat  as  represented  by  ancient  combat;  and  modern  com- 
bat characterized  by  the  introduction  of  weapons  with 
large  firepower. 

Letting  x(t)  and  y(t)  represent  the  opposing  force 
levels,  in  the  first  case,  the  attrition  depends  on  both 
force  levels  and  is  given  by 

||  =  -ax(t)y(t)  (1) 

|£  =  -bx(t)y(t)  (2) 

where  a  and  b  are  called  Lanchester  attrition-rate  coeffi- 
cients.  Dividing  Eq .  1  by  Eq.  2  and  integrating  yields 
the  force  level  equation: 

b(x(0)  -  x(t))  =  a(y(0)  -  y(t)) 

which  is  known  as  the  Lanchester  linear  law.   If  x  is  to 
win  (x(t)  >  0,  y(t)  =  0) ,  then  the  initial  force  levels 
must  satisfy 


x(0)  >  |  y(0) 

In  the  second  case,  the  attrition  depends  only  on  the 
opposing  force  level  and  is  given  by 

g|  -  -bx(t) 

Prom  this  system  of  equations, Lanchester ' s  square  law, 

b(x2(0)  -  x2(t))  =  a(y2(0)  -  y2(t)) 
is  derived  with  the  condition  for  x  to  win  that 

x2(0)  >  |  y2(0). 

Since  Lanchester 's  work,  numerous  extensions,  modifica- 
tions, etc.,  have  been  advanced.   A  survey  of  the  work  in 
this  area  can  be  found  in  Dolansky  [3]. 

An  obvious  drawback  to  the  Lanchester  formulations  is 
that  they  are  deterministic.   Given  the  attrition  coeffi- 
cient, the  winner  is  determined  by  the  initial  force  levels. 
Clearly,  this  is  not  true  in  actual  combat.   There  are  a 
great  many  other  factors,  which  may  be  considered  as  random 
variables,  which  enter  into  the  determination.   Apparently 
the  first  stochastic  formulation  was  given  by  Koopman  [^,5]. 

Weiss  [6]  noted  a  number  of  deficiencies.   Of  particular 
significance  is  the  fact  that  the  Lanchester  model  assumes 
that  all  units  on  a  side  are  of  the  same  type  (homogeneous 


forces).   In  actual  combat,  there  are  units  of  many  dif- 
ferent types  on  each  side  (heterogeneous  forces). 

The  simplest  heterogeneous  force  model  involves  a 
single  force,  Y,  facing  an  opposing  force  composed  of  two 
types  of  units,  Xj  and  X2 .   The  attrition  structure  is 
described  by  some  function  of  attrition  coefficients  and 
force  levels.   The  Y  force  commander  faces  an  allocation 
of  fire  problem.   How  should  he  divide  his  fire  between 
Xj  and  Xa? 

.  In  the  following,    deterministic  and  stochastic  models 
of  this  problem  are  developed  and  the  solution  obtained 
by  application  of  optimal  control  theory  is  given.   For 
particular  numerical  values,  the  optimal  allocation  of 
fire  and  the  terminal  pay-off  are  compared. 


Table  I.   Notation 

XjCt),  x,(t)9  y(t)  --  State  variables,  number  of  survivors 
of  Xla  X2,  Y  at  time  t. 

T   —  Specif ied 'maximum  duration  of  the  battle. 

p}  q,  r  —  Value  of  a  survivor  of  the  battle  of  Xp  X2,  Y 
respectively . 

a1  —  Attrition  coefficient,  rate  at  which  one  y  attrits  X} 

a2  —  Rate  at  which  one  y  attrits  X2. 

bj  —  Rate  at  which  one  xx    attrits  Y. 

b2  —  Rate  at  which  one  x2  attrits  Y. 

cj)   —  Decision  variable,  fraction  of  Y  fire  directed  at 
Xx  ,   0  <  cj>  <  1. 


R  = 


aibi 
a2b2 


a2p 

s  =  bjXj (0)  +  b2x2(0) 
R  -  s 


Z  = 


R  -  1 


A  =  R(Z2  -  1)  -  Z2 
(Z  -  l)2 

R(Z2  -  1)  -  Z2 


B  = 


Z2 


q  /  a2 


II.   THE  DETERMINISTIC  MODEL 

The  development  of  the  model  requires  three  major 
decisions.   An  objective  function  must  be  chosen.   A  stop- 
ping rule  must  be  defined  and  the  structure  of  the  attrition 
mechanism  selected. 

Among  the  Y  force  commander's  objectives  might  be  to 
maximize  casualties  inflicted,  minimize  the  casualties 
incurred  or  to  maximize  the  duration  of  the  battle.   The 
objective  function  used  in  this  model  is 

max  ry(T)  -  px  (T)  -  qx2(T) 

that  is,  the  Y  force  commander  desires  to  allocate  his 
forces  to  maximize  the  net  worth  of  the  survivors,  the 
terminal  pay-off.   Appropriate  choices  of  the. weighting 
factors,  p,  q,  and  r,  reflect  Y's  feeling  regarding  casual- 
ties.  At  one  extreme,  p  and  q  set  equal  to  zero  correspond 
to  minimizing  casualties  incurred,  while  r  set  equal  to 
zero  essentially  maximizes  casualties  inflicted.   It  can 
be  shown  that  maximizing  the  duration  of  the  battle  leads 
to  a  different  allocation  policy. 

There  are  three  ways  in  which  the  battle  may  end.   One 
force  may  decide  to  disengage,  either  as  a  function  of 
casualties  incurred  or  after  some  specified  time,  or  one 
force  may  be  driven  to  zero.   In  the  deterministic  formula- 
tion, there  is  a  one-to-one  correspondence  between  casual- 
ties and  elapsed  time. 
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The  following  stopping  rule  is  used  in  this  model. 
With  T  specified,  there  are  four  states  in  which  the  battle- 
can  end: 

Cj   y(T)  >  0  and  either  x^T)  >  0  or  x2(T)  >  0  or  both 
(time  expires ) . 

C2   y(t)  =  0  while  either  Xj(t)  >  0  or  x2(t)  >  0  or 
both  for  0  <  t  <  T  (Y  loses). 

C3  .XjCt)  =  x2(t)  =  0,  y(t)  >  0  for  0  <  t  <  T  (X  loses) 

C\   xx(t)  -  x2(t)  =  y(t)  =  0  for  0  5  t  <  T  (draw). 

States  C2  to  C^  are  referred  to  as  f ight-to-the-f inish  or 
terminal  control  battles,  while  C   is  a  prescribed  duration 
battle.   The  model  is  formulated  as  a  prescribed  duration 
battle  with  consideration  given  to  premature  terminations 
(C2,  C3,  or  Cj,   If  T  is  large  enough,  the  premature 
terminations  will  determine  the  actual  time  the  battle  ends 

The  attrition  structure  used  is  the  Lanchester  square 
lav;.   These  choices  lead  to  the  following  model. 

max   ry(T)  -  px2(T)  -  qx2(T)  (3) 

0<4><1 

dx 

ST   -rr-  =  -  Cf>aiy(t) 


dt 
dx 


-  =  -  (1-4,)  a.y(t)  (4) 


dt 

%     =  "  (b^Ct)  +  b2x2(t)) 
l>    x2,  y  >  0.      T  specified. 
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A  number  of  assumptions  are  involved  in  the  formulation 
and  use  of  the  model. 

Al.   Each  unit  has  perfect  information  regarding  the 
state  and  location  of  enemy  units. 

A2 .   Fire  may  be  shifted  instantaneously. 

A3.   Fire  is  uniformly  distributed  over  surviving  units 

kH .   All  enemy  units  are  within  range. 

A5.   The  effects  of  successive  rounds  are  independent. 

A6.   The  attrition  coefficients  ax,  a2  ,  b1}  b2  are 
constant  throughout  the  battle. 

A7 •   Fractional  casualties  are  allowed. 

A8.   a^bj  -  a2b2.   Since  the  labelling  of  forces  is 

arbitrary,  this  is  a  nonrestrictive  assumption. 

Al  through  A  5  are  inherent  in  the  Lanchester  square  law 
formulation,  as  developed  by  Barfoot  [7]. 

This  problem  was  first  formulated  and  partially  solved 
by  Isbell  and  Marlow  [8],  before  Pontryagin  announced  his 
maximum  principle.   Using  subsequent  developments  in  modern 
optimal  control  theory,  Taylor  [9,  10]  has  obtained  a 
complete  solution  to  both  the  fixed-duration  and  the  fight- 
to-the-f inish  battle. 
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III.   SOLUTION  TO  THE  DETERMINISTIC  MODEL 


The  derivation  of  Taylor's  solution  is  exceedingly 
lengthy  and  tedious.   The  results  are  summarized  in  Appen- 
dix A.  This  section  discusses  Taylor's  solution  technique 
and  the  insight  into  the  structure  of  the  optimal  allocaticr. 
policy  provided  by  this  solution. 

The  solution  procedure  begins  by  forming  the  Hamiltonian 


H 


(t,  x±i   p±3  <t>)   =  -  {Pj(t)  (j^y  +  p2(t)(l-<}>)  a2y 
+  p3(t)(blx1  +  b2x2)} 


(5) 


where  p  ,  p  ,  and  p   are  the  dual  variables  associated  wit] 
X1?  X2  and  Y  respectively. 

By  the  Pontryagin  maximum  principle  ,  maximizing  Eq.  5 
at  all  points  in  time  yields  the  maximum  of  Eq.  3-   Rewrit' 
ing  Eq .  5  as 

H   =    -    {(JjfajPjCt)    -   a2p2(t))    y   +   a2p2(t)    y 

+   p,(t)(blx1    +   b2x2) 


it  is  clear  that  the  optimal  control, 
f. 


is  given  by 


<f>*  = 


1  if  a2p2(t)  >  axpx(t) 
0  if  a2p2(t)  <  axps  (t) 


(6) 


This  paper  uses  the  American  form  of  the  maximum 
principle  [11].   Pontryagin,  et .  al.,  [12]  use  an  equivaler.- 
form  which  differs  in  sign  conventions. 
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It  can  be  shown  that  a2p2  =  dx]?1    cannot  hold  over  a  finite 
interval  of  time,  hence  the  optimal  control  is  to  always 
concentrate  fire  against  one  target  type. 

The  adjoint  system  of  differential  equations  for  the 
dual  variables  is  given  by 

dp  j 

at"  =  biP3^) 


dp 


2 


dt    =  b2P3(t)  (?) 

dp3 

^—  =  p1(t)a1<J>  +  p2(t)a2(l-(f)) 

When  the  optimal  value  function  is  dif f erentiable,  the 
dual  variables  represent  the  change  in  the  optimal  value 
function  caused  by  a  change  in  the  corresponding  state 
variable.   In  particular,  if  no  force  has  been  driven  to 
zero  at  the  end  of  the  battle,  the  final  values  of  the 
dual  variables  are  easily  found.   Differentiating  Eq.  3 
with  respect  to  the  state  variables  yields 

Pj(T)  -  -  p 

P2(T)  =  -  q 

p3(T)  =  r. 

If  either  X.  or  X2  is  driven  to  zero  during  the  course 
of  the  battle,  the  final  values  of  the  dual  variables  can 
no  longer  be  determined  as  above;   instead,  the  theory  of 
state  variable  inequality  constraints  must  be  applied. 
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Taylor  [Appendix  E  of  10]  presents  a  detailed  discussion 
of  this  theory. 

When  a  force  is  driven  to  zero,  the  optimal  trajectory 
is  said  to  enter  a  constraint  sub-arc.   The  point  of  entry 
is  called  a  corner.   In  this  problem,  a  nonnegativity 
constraint  is  binding  on  a  constrained  sub-arc.   One  treat- 
ment, developed  by  Gamkrelidze  [12]  is  to  modify  the 
Hamiltonian  by  adjoining  the  first  time  derivative  of  the 
constraints  to  the  criterion  functional  with  additional 
Lagrange  multipliers. 

H   =    -   Pja^y   -   p2a2(l-<J>)y   -   p3(b1x1    +  b£x2) 

-  y1(t)a1<j>y   -   u  2  ( t  )a2  (l-<j>  )y 


The  y.'s  are  multipliers  satisfying 


r=  0    x.  >  0 
i 

>  0    x.  =  0 

i 


as  well  as  Gamkrelidze ' s  condition 


i  =  1,  2 


dt 


<  0 


for  x.  =  0 
l 


There  are  also  corner  conditions  on  the  dual  variables 
at  an  entrance  to  a  constrained  sub-arc.   If  X   is  driven 
to  zero  at  t  ,  these  are 

P,(t;)   =  PjCtJ)   +  ya(t|) 


P2(t;)  =  Pa<t;> 
p,(t;)  -  p3(<) 
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In  addition,  the  Hamiltonian  is  continuous  across  the 
corner.   Approaching  the  corner  (tj),  <t>  =  l,    p  b  x  approache: 
zero,  and  u.=0.   After  the  corner  (t^,  <}>=0,  Xj  =  0,  and 
y2  =  0.   From  the  continuity  of  the  Hamiltonian 

H(t7)  =  -  p1(t7)a1y  -  P3b2x2 

=  -  p2(t1)a2y  -  P3b2x2  =  H(tx) 
therefore 


px(t«7)  = 


a2  + 

r-P2(tt) 


a 


p2(tT) 


On  a  constrained  sub-arc  with  Xj=0,  the  multiplier  is 
determined  from  the  condition 


dH* 
d<j> 


=  0 


which  yields 


Uj(t)  =  ~   (aap  (t)  -  axpj(t)) 


Since 


du 


1   f   dP 


dt 


'2dt 


-  a 


dpx" 


i  dt 


P,(t) 
a. 


(a  b   -  a  b  ) 

2   2        11 


using  the  adjoint  equations  (Eq.  7)  it  is  clear  that 
Gamkredlidze ' s  condition  is  only  satisfied  on  the  con- 
strained sub-arc  x  =0  if  ajb,  >    a2b2  since  it  is  readily 
shown  that  p,(t)  >  0. 
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A  similar  argument  shows  that  Gamkrelidze ' s  condition 
is  satisfied  on  the  constrained  sub-arc  x2=0  only  if 
a  b   <  a  b0  which  contradicts  A8.   Therefore,  it  is  optimal 

112   2  ' 

to  drive  Xz  to  zero  only  at  the  end  of  the  battle,  if  at  all 

Since  the  values  of  the  dual  variables  are  known  for 
t=T,  it  is  convenient  to  define  the  backwards  time  variable 

T=T-t  0  <:  t  <  T 

and  so 

dr-  :  dt~  dr  :  "  dF~      1  ~  >      '    5  K    } 

Substitution  of  Eq .  8  into  Eq .  7  yields  a  system  of  differ- 
ential equations  with  known  initial  conditions  which  can 
be  solved  for  the  dual  variables  as  a  function  of  time. 

Once  the  dual  variable  functions  are  found,  the  optimal 
control  is  given  by  Eq.  6.   The  time  at  which  fire  is 
switched  from  Xj  to  X2  is  found  from 

axpj (t)  =  a2p2(t) . 

The  trajectories  of  the  state  variables  are  easily 
determined.  For  an  interval  (t1}  t2)  where  4>  =  1,  Eq .  4 
becomes 


x1(tl)    given 

x2(tj)  given 
y(tj)  given 


dx  j 

dt 

— 

axy(t) 

dx2 
dt 

0 

dt 

- 

(blXl( 
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which  can  be  solved  to  get  for  tj  1  t  1  t2 

'  (bjX^Ctj)  +  b2x2(t2)) 


Xj(t)  =  XjCtj)  +  af 

y(t2) 


axb, 


cosh  ( /a 1 b  x  t ) 


/aibi 


sinh  ( /a 1 b : t )  - 


(bjXjCtj)  +  b2x2(t1)) 


a1b1 


x  (t)  =  x2(t1) 


(9) 


y(t)  =  y(tx)  cosh  ( /a  j  b  j  t ) 


sinh  ( /a  j b  x  t ) . 


/ajb ! 

Similarly,  for  an  interval  (t3,  tu)  where  (J)  =0,  Eq .  4 
becomes 


dx 


dt 


dx 
dT 


=  0 


a.y(t) 


|f  =  -  (bjXjCt)  +  b2x2(t) 


dt 


which  gives  for  t   £  t  £  t 


Xj(t3)  given 


x2(t3)  given 


y(t3)  given 


x  j  ( t )  =  x ,  ( 1 3 ) 
x2(t)  =  x,(t,)    a,- 


'(b,x,(tj  +  b  x  (t  )1 

^   1   1  v   3        2   2    3' 


2  v  "  3 


a2b2 


cosh  ( /a , b , t ) 


2^2 


y(tj 


^K 


sinh  (/a2b2  t)  - 


(b1x1(t3)  +  b2x2(t3)) 


a2b2 
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y(t)  =  y(t3)  cosh  (/a7E7  t) 
(b1xl(t3)  +  b2x2(t3)) 


/a2b2 


sinh  (/a2b2  t)    .     (10) 


Neglecting  drawn  battles  (terminal  state  C  ,)  the 
terminal  states  can  be  partitioned  as  shown  in  Table  II. 
Using  the  optimal  control  determined  by  Eq.  6,  the  state 
equations,  Eqs.  9  and  10  can  be  integrated  in  forward  time 
to  determine  what  combinations  of  parameters  and  initial 
force  levels  can  lead  to  particular  terminal  states. 
These  conditions  define  the  domain  of  controllability. 

Two  additional  items  must  be  checked.   First,  the  do- 
mains of  controllability  must  cover  the  entire  state  space. 
If  not,  then  there  exists  a  singular  surface  where  the 
Pontryagin  maximal  principle  does  not  apply  and  other 
methods  must  be  used  to  determine  the  optimal  control. 
Such  a  surface  is  not  present  in  this  problem. 

Secondly,  it  may  be  that  the  domains  of  controllability 
overlap.   If  this  is  the  case,  then  the  optimal  control  is 
the  one  leading  to  the  terminal  state  with  the  largest: 
pay-off.   Unless  one  terminal  state  dominates  the  other, 
the  region  of  overlap  is  partitioned  by  a  dispersal  surface 
into  subregions  from  which  optimal  extremals  lead  to  a 
particular  terminal  state. 

Dispersal  surfaces  are  present  in  this  case.   The 
resolution  of  these  areas  can  be  found  in  Taylor  [10].   As 
this  paper  is  primarily  concerned  with  numerical  results, 
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Table  II.   Terminal  States 

State        Y            X,  X2           T, 

>0  >0           =T 

=0  >0           =T 

>0  =0           =T 

>0  >0           IT 

=0  >o         It 

>0  =0           <T 

=  0  =0            <T 

=0  =0           <T 

T  :   Specified  duration  of  the  battle. 

Tj!   Time  battle  actually  ends. 

SI  -  S3   partition  Cl    (fixed-duration  battle) 

S4  -  SG   partition  C2  (Y  loses) 

S7  -  S8   partition  C3  (X  loses) 


SI 

>0 

S2 

>0 

S3 

>0 

S4 

=  0 

S5 

=  0 

S6 

=  0 

S71 

>0 

S82 

>0 

Xx  destroyed  first. 

2 

X2  destroyed  first. 


the  question  is  resolved  by  computing  the  pay-off  for  each 
possible  terminal  state  and  choosing  the  maximum. 

Insight  into  the  allocation  policy  can  be  gained  by 
considering  the  interpretation  of  the  quantities  a1b1, 
a2b2,  ajP,  and  a2q.   The  quantity  a1b1(a2b2)  is  a  strategic 
or  long  term  measure.   It  can  be  interpreted  as  a  measure 
of  the  rate  at  which  Y  destroys  X^s  (X2's)  ability  to 
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destroy  Y.   On  the  other  hand,  a1p(a2q)  is  a  tactical  or 
short  term  measure,  interpreted  as  the  rate  at  which  Y 
destroys  X  's  (X2's)  value.   By  A8,  aJb1  >  a2b2,  so  there 
are  two  cases  to  consider. 

If  a.p  >  a2q,  the  situation  is  relatively  simple. 
Y  receives  more  return  in  both  the  strategic  and  tactical 
sense  by  firing  at  X,,  hence  it  is  never  optimal  to  fire 
at  X2  unless  Xx  =  0.   Accordingly,  in  this  case,  no 
extremals  lead  to  states  S3,  S6,  or  S8.   For  all  other 
terminal  states,  <J)*  =  1  until  X^O  or  Y=0. 

Things  are  more  complicated  if  a2p  <  a2q,  since  Y 
obtains  a  better  tactical  return  by  firing  at  X2,  but  a 
better  strategic  return  by  firing  at  X  .   At  the  termina- 
tion of  the  battle,  only  the  tactical  return  need  be  con- 
sidered, so  clearly,  <J>*(T)  =  0  as  long  as  there  are  any  X2 
survivors  at  T. 

However,  since  Y  is  more  effective  in  destroying  X^s 
kill  capability  than  X2's,  it  is  in  Y's  best  interest  to 
fire  at  Xj  until  there  is  just  enough  time  remaining  to 
destroy  X2  then  shift  fire.   Otherwise,  Y  will  incur  need- 
less casualties. 
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IV.   THE  STOCHASTIC  MODEL 

A.   STOCHASTIC  CONTROL  PROBLEMS 

The  addition  of  stochastic  elements  to  the  optimal 
control  problem  Introduces  several  complications.   There 
are  a  number  of  aspects  which  must  be  considered,  since 
different  interpretations  lead  to  different  courses  of 
action.  '  These  are  discussed  at  length  by  Dreyfus  [13]. 

The  state  of  knowledge  of  the  underlying  stochastic 
process  must  be  considered.   In  the  simplest  case,  the 
properties  of  the  process  may  be  known.   On  the  other  hand, 
if  the  nature  of  the  process  is  not  known,  a  learning 
element  enters  the  problem.   The  course  of  action  chosen 
must  be  refined  as  the  process  evolves  and  knowledge  is 
gained.   This  is  known  as  an  adaptive  control  problem. 

Another  important  factor  is  the  ability  to  observe 
the  state  of  the  system.   In  the  simplest  case,  the  state 
can  be  observed  precisely  at  any  particular  point.   If  the 
state  is  only  partially  observable,  then  an  estimation 
problem  must  also  be  treated. 

Since  the  trajectory  is  no  longer  precisely  determined, 
the  objective  function  becomes  a  random  variable-.   Among 
the  alternatives  for  treating  the  objective  function  are: 
the  maximization  of  an  expected  value;  maximizing  the 
probability  of  reaching  a  particular  state;  or  minimizing 
the  variance  of  the  terminal  state. 
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A  significant  difference  from  the  deterministic  problem 
is  that  the  manner  in  which  the  control  is  applied  leads 
to  different  results.   There  are  three  basic  control  poli- 
cies to  be  considered:   open  loop;  feedback;  and  open 
loop-feedback . 

An  open  loop  control  policy  represents  the  case  in 
which  the  control  must  be  specified  before  the  process 
starts  and  cannot  be  altered.   In  a  process  evolving 
through  time,  the  control  is  expressed  as  a  function  of 
time,  given  the  initial  state. 

A  feedback  or  close  loop  policy  assumes  the  ability 
to  continuously  observe  the  state  of  the  system  and  modify 
the  control.   Mathematically,  the  control  is  expressed  as 
a  function  of  time  and  state,  given  the  initial  state. 

The  open  loop-feedback  policy  arises  if  it  is  impossible 
to  continuously  observe  the  system.   Generally,  the  system 
can  be  observed  only  at  particular   points  in  time.   In 
this  case,  an  open  loop  control  based  on  the  initial  state 
is  applied  to  start  the  process.   After  an  interval  of  time, 
the  state  is  observed  and  another  open  loop  control  is 
applied,  based  on  the  observed  state.   This  continues  until 
the  process  terminates. 

It  is  fairly  obvious  that  the  feedback  policy  is  the 
"best"  of  the  three.   The  process  can  be  rigidly  controlled 
since,  should  the  process  start  to  behave  in  a  nonoptimal 
fashion,  it  can  be  corrected  immediately.   In  an  open  loop 
situation,  a  more  delicate  touch  is  required.   There  is  no 
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way  to  correct  any  over-control.   The  open  loop-feedback 
corresponds  to  the  physical  situation  usually  faced  in  the 
actual  problem,  as  opposed  to  the  model. 

In  the  following  section,  a  feedback  control  will  be 
found  for  a  model  where  the  underlying  process  is  known 
and  the  states  are  fully  observable. 

B.   EXACT  SOLUTION  OP  THE  STOCHASTIC  MODEL 

This  model  differs  from  the  deterministic  model  in 
the  structure  of  the  attrition  mechanism.   The  physical 
situation  is  the  same  as  in  Section  II.   Assumptions  Al 
through  A6  and  A8  are  retained.   A7  is  replaced  by 

A7 '  .   Only  integer  casualties  can  occur  and  Y  can 
allocate  only  integer  numbers  of  troops. 
The  objective  function  is 

MAX  E{ry(T)  -  px^T)  -  qx„(T)) 

This  model  treats  the  prescribed  duration  battle.   For 
reasons  similar  to  those  of  the  deterministic  model,  it  Is 
convenient  to  work  in  backwards  time,  t  =  T-t  where  T  is 
the  prescribed  termination  time. 

Instead  of  a  deterministic  square  law  attrition  mecha- 
nism, the  attrition  is  modelled  as  a  stationery  Markov 
process.   The  following  probability  statements  are  assumed 
for  a  sufficiently  small  increment  of  time  Ax: 

A9 .   Prob  [  one  Xi  casualty  in  Ax]  =  a.y^Ax 

Prob  [  more  than  one  X1    casualty  in  Ax]  =  o(Ax) 
Prob  [  one  X2  casualty  in  Ax]  =  (l-<{>)  a2yAx 
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Prob  [more  than  one  X2  casualty  in  Ax]  =  o(At) 

Prob  [one  Y  casualty  in  At]  =  (b  xx  +  b2x2)AT 

Prob  [more  than  one  Y  casualty  in  At]  =  o(At) 

Prob  [more  than  one  casualty  of  any  type  in  At] 
=  o(At) 

where  o(At)  is  a  quantity  such  that  lim  — — — -  =  0.   Given 

At+0 
the  above,  it  follows  that 

P    [no   Xj    casualties    in   At]    =    1   -    <j>a   vAt   +   o(At) 

and  similarly  for  X2  and  Y. 

The  assumption  that  Y  has  perfect  information  about 
the  state  of  the  system  at  any  time  and  the  ability  to 
instantaneously  shift  fire  leads  to  a  feedback  or  closed 
loop  optimal  control. 

To  solve  this  model,  the  optimal  expected  value  function, 
S,is  defined  as 

S[t,Xi(t),  x2(t),  y(T)]  =  E[ry(0)  -  px^O)  -  qx2(0)] 

given  that  the  state  at  time  t  is  x^t),  x2(t)  and  y(T) 

and  that  an  optimal  control  has  been  applied  in  the  interval 

[0,t]. 

Table  III  lists  five  .events  which  could  take  place  in 
an  interval  of  length  At,  [t,  t-At]  and  their  probability 
of  occurrence.   An  expression  for  S(t,  x  ,  x  ,  y)  is 
developed  by  the  application  of  Bellman's  Principle  of 
Optimality  [l'J]  which  states: 
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Table  III.   Transition  Probabilities 


State   @t 

Casualties 

Probability 

Id    x29   y 

none 

1   -    (^a2y   +    (l-<J0a2y   +  b!xi 
+   b2x2}AT 

Xj-la    x2,    y 

one    X 

cj^yAT 

xlS    x2-l,    y 

one   X2 

(l-(f))a2yAT 

xlS    x2,    y-1 

one   Y 

(bjXj    +   b2x2)AT 

Various 

more    than   1 

o(Ax) 

An  optimal  policy  has  the  property  that, whatever  the 
initial  state  and  initial  decision  are,  the  remaining 
decisions  must  constitute  an  optimal  policy  with  regard 
to  the  state  resulting  from  the  first  decision. 


By  definition,  the  return  from  any  of  the  states  listed  in 
Table  III  resulting  from  following  an  optimal  policy  to 
the  end  of  the  battle  is  S(t-At,  xx,  X2,  y).   Therefore, 
the  optimal  value  of  <\>    in  the  interval  [t,  t~At]  is  the 
one  which  maximizes  the  return  expected,  starting  from 
state  xls  x2,  y  at  time  t.   This  can  be  expressed  as 

S[t,  x.,  x  ,  y]  =  max  {[1  -  {<|>a  y  +  (l-<f>)a  y 

0<<f><l 

+  b,x,  +  b^x„}Ai]S(T-AT,x1 ,x2,y) 


i  i 


2   2 


+  (j>a1yAxS(T-AT,x1-l,x2  ,y) 


+  (l-c())a2yATS(T-AT,x1,x2-l,y) 


(11) 


(b1x1  +  b2x2)ATS(T-Ax,xl ,x2,y-l)  +  o(At)} 
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Since  S(t-At,  x15  x2,  y)  does  not  depend  on  the  control 
variable,  <J> ,  this  can  be  rewritten  as 

S(Ta  xlS  x2,  y)  -  S(t-At,  xl5  x2,  y) 

=  max    {(bjXj    +   b2x2 ) [S(t-At ,    xx,    x2,    y-1) 
0<<J><1 

-S(t-At,    xis    x2,    y)] 
+   yf^a^SCi-AT,    Xj-lj    x2,    y)    -   S(t-At,    x19    x2,    y)] 
+    (l-<j))a2[S(T-AT,    xls    x2-l,    y)    -   S(t-At,    xx,    x2,    y)] 

4.     POT)       1 

In   the    limit   as    At-*03 

dS(x ,    x13    x2 ,    y) 

^ =    (b1x1    +   b2x2)[S(T,    Xj.,    x2,    y-1) 

-    S(t,    Xj,    x2,    y)] 

+   y  max   {<$>(&    [S(t,    x  -1,    x    ,    y)    -   S(t,    x    ,    x2  ,    y)] 
0<<J><1 


-   a 


[S(t,    xx,    x2-1,    y)    -   S(t,    xl9    x2,    y)])       (12) 


+   a2[S(T,    x    ,    x2-l,    y)    -   S(t,    xx  ,    x2 ,    y)] 
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Equation  12  can  also  be  obtained  directly  from  Kushner's 

general  result  on  the  optimal  control  of  a  "Poisson" 

2 
process  [15] • 

To  determine  $,  the  switching  function 

w(t)  =  a1{S(T,  Xj-1,  x2  ,  y)  -  S(t,  xx  ,  x2  ,  y)} 

-a2(S(T,  Xj,  x2-l,  y)  -  S(t,  xx   ,  x2  ,  y)}   (13) 

is  used.   Equation  12  then  becomes 

dS(x,    xp    x    ,    y) 

3 =    (b    x      +  b    x    ){S(T,    x    ,    x    ,    y-1) 

dx  11  2      2       L  L »  1 '  2 '      J 

-   S(T,    xx ,    x2,    y)} 
+   a2y{S(T,    xj5    x2-ls    y)    -   S(T,    x2 ,    x2 ,    y)f 

+   y  max    { 4>  v/  (  t  ) } 

0<cf)<l 

Clearly,  the  optimal  control  is  given  by 
fl    w(t)  >  0 

<j)*(T)  = 

[0    w(t)  <  0 

In  the  deterministic  case  where  the  optimal  control 
depended  on  the  dual  variables,  it  was  necessary  to  treat 
the  case  where  a  force  is  driven  to  zero  separately  since 


2 

This  contains  a  typographical  error.   The  unnumbered 

equation  after  Eq .  7  should  read 

n        n 
0  =  min  {k  +  V  +  I    p  g   +  I    c,[V(x+X    t)  -  V(x,  t)]} 
u        L    1   1  1    l   x      x 

If  i  =  l,  this  reduces  to  Dreyfus  result  [13]. 
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the  boundary  influenced  the  solution  for  the  dual  varia- 
bles.  In  the  stochastic  case,  the  optimal  control  depends 
only  on  the  optimal  expected  value  function.   As  shown 
below,  the  initial  conditions  for  S  are  easily  determined 
even  when  a  force  is  driven  to  zero.   Therefore,  no  special 
distinction  between  the  fixed  duration  and  the  fight  to 
the  finish  battles  is  necessary  to  solve  for  the  optimal 
expected  value  function. 

If  t=0  (i.e.  at  the  end  of  the  battle),  then 

S(0,  xl?  x2,  y)  =  ry  -  px>  -  qx2  (llj) 

for  all  values  x, ,  x2,  and  y. 

If  y=0,  there  is  no  battle  and 

S(t,  xx,  x2,  0)  =  -  (pxj  +  qx2)  (15) 

for  all  values  of  t,  x  ,  and  x2 . 

Similarly,  if  xx  =  x2  =  0,  there  is  no  battle  and 

S(t,  0,  0,  y)  =  ry  (l6) 

for  all  values  of  t  and  y. 

IfXj  =  0,  then  clearly,  <J>  =  0  for  all  t  and  Eq .  12 
reduces  to 


dS(x,  0,  x,,  y) 


2 


=  b  x  rs(T,  0,  x,,  y-l)  -  S(t,  0,  x,y)] 


dl  2   2  L   v   *    >    2 


+  ya2[S(i,  0,  x2-l,  y)  -  S(x,  0,  x2,  y)] 


with  the  initial  condition  from  Eq .  14 


S(0,  0,  x2,  y)  =  ry  -  qx2  . 
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In  particular,  for  x2  =  y  =  1 


dS(T,  0,  1,  1) 
dx 


=  b2[S(T,  0,  1,  0)  -  S(T,  0,  1,  1)] 


+  a2[S(x,  0,  0,  1)  -  S(x,  0,  1,  1)] 


=  b2[-q-S(T,  0,  1,  1)]  +  a2[r-S(x, 0,1,1)] 


using  Eqs.  15  and  16  or 


dS(x,  0,  1,  1) 
dx 


=  -  (b,  +  a,)S(T3  0,  1,  1)  +  a,r  -  b2q 


with  S(0,  0,  1,  1)  =  r  -  q.   This  is  an  ordinary  differen- 
tial equation  of  the  form 


dx  _ 

dt  "  cix  +  c2 


which  is  easily  solved  to  give 


S(x,  0,  1,  1) 

fa2r  -  b2q' 


fbjr  -  a?V 


a2  +  b2 


exp  {-(a2  +  b2)x) 


a,  +  b0 

2       2 


(17) 


Similarly,  if  x2  =  0,  then  <j>  =  1  for  all  t  and  for 
x,  =  y  =  lj  Eq.  12  reduces  to 

dS(T,  1,  0,  1) 


dx 


b^-p-Sd,  1,  0,  1)}  +  a^r-SCx, 1,0,1)} 


=  ■-  (bj  +  ai)S(T,  1,  0,  1)  +  axr  -  bxp 


with  S(0,  1,  0,  1)  =  r  -  p.   The  solution  in  this  case  is 
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(b 


S(t,  1,  0,  1) 


ir  -  aiP 
ai  +  bi 


exp  j-(aj  +  bl )t} 


a,r 


biP] 


ai  +  bi 


(18) 


Letting  xx  =  x2  ■  y  =  1,  Eq.  12  becomes 

dS(T?  ^  1?  1}  =  (bx  +  b2)[S(x5  1,  1,  0)  -  S(t,  1,  1,1) 


+  a,[S(i,  1,  0,  1)  -  S(t,  1,  1,  1)] 


+  max  Ufa ,[S(t,  0,  1,  1)  -  S(t,  1,  1,  1)] 

0<<|><1 

-  a2[S(i,  1,  0,  1)  -  S(t,  1,  1,  1)])}  .      (19) 


The  switching  function  determines  $ .      At  the  end  of 
the  battle 

w(0)=a1(r~q-r+p+q)  -a2(r-p-r+p+q) 

=  axp  -  a2q 

so  there  are  two  cases  to  consider. 
1.   Case  I 


axp  Z  a2q 


4>*(x)  =  1  for  0  <  t  5  ij  where  x,  is  the  time  when  Y 
switches  fire  from  X.  to  X2,  determined  by 

w(xx)  =  0. 

This  is  consistent  with  the  interpretation  of  a   p   as  a 
tactical  measure.   Certainly,  at  the  end  of  the  battle, 
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Y  should  direct  his  fire  to  get  the  greater  return.   In 
this  case,  Eq .  19  becomes 


dS(x,  1,  l,  l) 
di 


(bx  +  b2){S(T,  i,  l,  0)  -  S(t,  1,1,1)} 


+  a1{S(x,  0,  1,  1)  -  S(t,  1,  1,  1)} 


with  the  initial  condition 


S(0,  1,  1,  1)  =  r  -  p  -  q. 


Substituting  Eqs.  15  and  17,  the  solution  is  given  by 


S(t,  1,  1,  1)  = 


aj (b2r  -  a2q) 
(a,  +  bL  -  a2)(a2  +  b2)  exp  <"  ^2+b2)T) 


[  ( b j  -  a2)(b1  +  b2)  +  a  j  b  j ] r        a i p 
j"Ta7"+  bj  -  a2)(aj  +  b1    +   b2)  "  (a.l    +   bx  +  b2) 


a  a  q 

1   2M 


+  (ax  +  bl    -  a2)(a1  +  b;  +  b2) 


;xp  {  -  (ax  +  b1    +   b2)x}  + 


axa2r 


(a2  +  b2)(a3  +  bx  +  b2) 


(bj  +  b2)p     [(b:  +  b2)(a2  +  b2)  +  a^Dq' 
(a1  +  b\    +  b2)      (a2  +  b2)(a1  +  b1    +  b2) 

for  0  <  T  <  t  . 
i 

Using  this  in  Eq .  13,  the  switching  time  x,  is  deter- 
mined.  Depending  on  the  value  of  T,  it  may  be  that  t , >T 
and  so  no  switch  occurs  during  the  battle.   If  t  <T,  then 
in  the  interval  Tj  1  t  1  t,  where  t2  is  the  time  of  switch 
from  X2  back,  to  X1?  (w(x2)  =  0)  ,  <|>*(t)  =  0  and 
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S(T,  1,  1,  1)  = 


a2(b2r  -  ajp) 


(a2  +  b2  -  a. 


)(a  +  b  )  f  exp{-(a1+b1)T} 


-  exp  {  (a2  +  b2)(x1  -  x)  -  a^  -  b^} 


a}a2r 
So  '  (ax  +  b1)(a2  +  b2  +  bx) 


[(bj  +  b2)(a:  +  bj)  +  a2b2]p      (b l    +  b2)q 
+   (Tj  +  b1)(a2  +  b"2  +  bj)    +  (a2  +  b2  +  b2) 


a1a2r 


exp    {-(a2    +  b2   +   bx)(x   -   Tl)J    +        (ai+bi)(a2+b2Tb7J 


C(bx  +  by)  (a,  +  bx)  +  a2bj]p     (b l    +  b2)q 


(ai  +  bi)(a2  +  b2  +  bi} 


(*2  +  b2  +  V 


where  S0  =  S(x  ,  1,  1,  1). 


2.   Case  II 


a„q  >  a,p 


In  this  case,  cf>(x)  =  0  for  0  f  x  1  x  x  where  x   is  the 
time  that  fire  switches  from  X2  to  X2.   For  this  interval 

O  (   Y)         T1         —         r^  X~)     ) 

S(x,   1,    l5    1)   =   (a2  +  ^  la.Xa,   +  b,)   exp   C-Ca.+b^x} 


[(b2    -   a1)(b1    +   b2)    +   a2b2]r 
(a2   +  b2   -aj )  (a2   +   b2   +   bl) 


a  2a2p 
+    (a2   +   b2   -   a1)(a2   +  b2   +   bx) 


a,q 


(a2    +   b2    +   bj) 


:xp{  -      (a2   +  b2  )x} 
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aia2r 


(a^Ma.+b.+b^ 


(.bj.+b^Jq 

TiT+VTbTT 


If   t^T,    then   for    tx    <   t    <   t 


ax (b2r-a2q) 


C(b1+b2)(a1+b-1)    +  a2bj] 

(ai+blHa2+t)2+bl) 


S(T, 1,1,1)  -  (i;+b1-a2)(a2+b2) 


exp  {-(a,+b,)T> 


-  exp  {(a1+b1)(T1-x)-a2T1-b  t} 


aia2r 


(b  +b  )p 

1    2  '  ^ 

o  "  Ti7+b1+b2)(a2+b2)  +  (a^b^bj 
[(bl+b2)(a2+b2)+arb2]q 


(a1+b1+b2)(a2+b2) 


exp    { (a1+b1+b2) (tj-t) } 


+   ' 


axa2r 


(b1+b2)p 
(a1+b1+b7)(a2+b2)    "    (a1+b1+b2) 


[(bl+b2)(a2+b2)+(a1b2)]q 

U1+b7+b~2lTa2+b2) 

S   =  S.(tj, 1,1,1)  where  t2  is  the  time  fire  is  shifted  back 
to  X2. 

With  S(t, 1,1,1)  known,  there  is  no  mathematical  reason 
why  Eq.  11  cannot  be  solved  for  S(t,2,1,1),  S(t, 1,2,1), 
etc.  in  hopes  that  eventually  a  general  solution  for 
SdjXjjX  ,y)  might  emerge.   For  the  stochastic  model  corre- 
sponding to  the  linear  law  attrition  process,  such  a  general 
solution  has  been  obtained  by  Clark  [16]. 
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No  general  solution  for  this  model  is  known.   From 
the • complexity  of  the  expression  for  S(t, 1,1,1),  it  is 
obvious  that  extensions  will  quickly  become  very  unwieldy. 
A  numerical  approximation  for  S(t,x  ,x  ,y)  appears  to  be 
a  more  profitable  line  of  approach. 

C.   NUMERICAL  APPROXIMATION  OF  THE  STOCHASTIC  MODEL 

In  the  development  of  the  differential  difference 
equation  for  S(x ,x1 ,x  ,y ) ,  a  suitable  numerical  approxima- 
tion was  also  developed. 

S(t+At,x. ,x_ ,y)    =  max      \    (1   -    {<J>a   y   +    (l-<J>)a   y 

0<<f><l    I 

+   bjXj    +   b2x2}AT)S(T,x1  ,x2  ,y)    +   ^a^AiS  (t  ,xl  -1  ,x2  ,y  ) 

+    (l-(J))a2yATS(T,xl  ,x2-l,y) 

+    (b1x1    +   b2x2)AiS(x,x1,x2,y-l)    j  (11) 

with  $  determined  by  the  value  of  the  switching  function 
w(x )  in  Eq.  13 . 

This  is  essentially  the  Euler  method  of  numerical 
approximation.   It  has  the  advantage  that  as  At-*-0,  the 
value  computed  above  will  converge  to  the  true  value  of 
S(t,Xj ,x2,y) . 

To  obtain  a  reasonable  approximation,  the  value  of 
the  time  step,  Ax,  must' be  small.   There  is  also  an  addi- 
tional restriction  on  At.   In  order  for  the  probability 
statements  assumed  in  A9  to  be  meaningful,  At  must  be  such 
that 
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axyAx  i  1 


a2yAi  <  1 


(b1xl  +  b2x2)Ax  <  1 


for  all  values  of  x,  ,  x£  and  y  considered. 

There  are  several  special  cases  of  Eq .  11 

S(0,xl5x2,y)  =  ry  -  pXj  -  qx2 


S(t,x1 ,x2,0)  =  -  px1  -  q 


x 


2 


S(T,0,0,y)  =  ry 
S(T+Ax,0,x2,y)  =  S(T,0,x2,y) 

+  AT(b2x2{S(T>0>x2Sy-l)  -  S(T,0Jx2,y) 

since  <j>  =  1  in  this  case  and 

S(t+At,x1 ,0,y)  =  S(x,xlS0,y) 

+  Ailb^^Sd^x^O^-l)  -  S(x,xi,0,y)} 

+  ajyJSCt.Xj-l.O.y)  -  S(x,x1 ,0,y ) }) 

since  <J)  =  0  in  this  case. 

The  above  equations  were  programmed  in  Fortran  IV  G 
and  run  on  an  IBM  360/67.   The  output  is  a  table  which 
gives  S  as  a  function  of  t  for  fixed  values  of  x  ,  x2, 
and  y . 

In  the  special  case  of  S(t, 1,1,1),  the  accuracy  of  the 
approximation  was  checked  against  the  exact  solution 
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developed  in  Section  III  B.   The  results  of  one  such 
comparison  are  given  in  Table  IV  .   It  can  be  seen  that 
the  approximation  agrees  v/ith  the  exact  solution  to  the 
fourth  decimal  place. 

For  a  given  force  level,  the  approximation  must  first 
calculate  the  values  of  S  for  all  combinations  of  smaller 
force  levels.   For  larger  force  levels,  this  requires 
a  large  amount  of  computer  time  and  memory.   For  a  rela- 
tively large  time  step  of  At  =  .01,  computation  of 
S(t,535j5)  for  0  1  x  <  50  required  about  four  minutes  and 
200  K  of  core.   To  solve  the  same  problem  for  S(t,9,9,9) 
would  require  about  one  hour  and  500  K.   Therefore,  the 
numerical  results  from  this  particular  approach  are 
limited  to  relatively  small  force  levels. 
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Table  IV.   Comparison  of  Exact  Solution 
and  Numerical  Approximation 

x  _                    Exact              Approximation 

in  minutes  S(t31,1,1) 

0  -.2000  -.2000 

1  -.2018  -.2018 

2  -.2037  -.2037 

3  -.2055  -.2055 

4  -.2073  -.2073 

5  -.2091  -.2091 

6  -.2108  -.2108 

7  -.2126  -.2126 

8  -.2143  -.2143 

9  -.2161  -.2161 

10  -.2178  -.2178 

11  -.2195  -.2195 

12  -.2212  -.2211 

13  -.2229  -.2228 

14  -.2245  -.2245 

15  -.2262  -.2261 

16  -.2279  -.2278 

17  -.2295  -.2294 

18  -.2311  -.2310 

19  -.2327  -.2326 

20  -.2343  -.2342 

21  -.2358  -.2357 

22  -.2374  -.2373 

38 


Table  IV.   (Continued) 


T 

in  minutes 

23 

24 

25 

26 

27 

28   , 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

4o 

41 

42 

43 

44 

45 

46 

Exact 
S(t, 1,1,1) 

-.2390 

-.2405 

-.2420 

-.2435 

-.2450 

-.2465 

-.2480 

-.2495 

-.2509 

-.2524 

-.2538 

-.2552 

-.2566 

-.2580 

-.2594 

-.2608 

-.2622 

-.2635 

-.2649 

-.2662 

-.2675 

-.2688 

-.2701 

-.2714 

Approximation 

-.2388 
-.2404 
-.2419 
-.2434 
-.24^9 
-.2464 
-.2479 
-.2493 
-.2508 
-.2522 
-.2537 
-.2551 
-.2565 
-.2579 
-.2593 
-.2606 
-.2620 
-.2633 
-.2647 
.  -.2660 
-.2673 
-.2686 

-.2699 
-.2712 
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Table  IV.   (Continued) 


t  Exact  Approximation 

in  minutes 

47 
48 

49 
50 


a,  =  .005       a2  =  .003 

p   =  .15        q   =  .45        r   =  .40        At  =  .01 


Exact 
S(t, 1,1,1) 

-.2727 

-.2739 

-.2752 

-.2764 

b,   = 

.007 

r      = 

.40 

-.2725 

-.2737 

-.2750 

-.2762 

b2 

=    .001 
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V.   COMPARISON  OF  RESULTS 

For  the  125  combinations  of  force  levels  from 
x   =  x   =  y  =  1  up  to  and  including  Xj  =  x2  =  y  =  53 
calculations  were  carried  out  for  each  of  the  four  sets 
of  parameter  values  listed  in  Table  V.   A  time  step  of 
.01  minutes  was  used  in  the  numerical  approximation. 
Comparisons  of  selected  results  are  given  in  Tables  VI 
through  XIV. 

These  tables  were  constructed  by  extracting  the  force 
level  time  history  from  the  deterministic  model.   Since 
the  deterministic  model  calculates  force  levels  as  contin- 
uous functions  of  time,  there  is  a  question  as  to  exactly 
when  a  casualty  occurs.   For  these  tables,  deterministic 
force  levels  are  rounded  up  to  the  next  integer,  i.e. 
Xj  =  3.001  is  rounded  to  xx  =  4.   With  these  values  of 
time  and  force  levels,  the  optimal  expected  value  function 
and  the  value  of  <f>*  were  obtained  from  the  appropriate 
stochastic  table.   All  comparisons  are  for  the  case  ajP<a2q 

In  the  case  a  p  >  a2q,  calculations  were  carried  for 
the  above  force  level  combinations  with  parameter  set  1. 
In  both  the  stochastic  and  deterministic  model,  there  is 
no  shift  of  fire  unless  x   is  driven  to  zero. 
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Table  V.   Parameter  Values  for 
Numerical  Calculations 


et 

ai 

a2 

b 

i 

b 

2 

P 

q 

r 

T 

1 

.005 

.003 

.007 

.001 

.15 

.45 

.4 

50 

2 

.005 

.003 

.007 

.001 

.15 

.45 

.4 

100 

3 

.025 

.015 

.035 

.005 

.75 

2.25 

2.0 

50 

4 

.05 

.03 

.07 

.01 

1.5 

4.5 

4.0 

50 

Table  VI.   Comparison  for  Xj(0)  =  x2(0)  =  y(0)  =  1 

with  Parameter  Set  2 

Time    Force  Level  S(t,x1}x2,y) 

in  minutes   x1   x2   y   Deterministic  Stochastic 

0  111  1 

49.94  111  1 

56.38         111  0 

100  111  0 

Deterministic  Pay-off   -.4127 
Terminal  State  SI 


1 

-.3175 

0 

-.2762 

0 

-.2686 

0 

-.2000 
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Table  VII.   Comparison  for  xl(0)    =  x2(0)  =  y(0)  =  5 

with  Parameter  Set  2 

S(t,Xj 3x2,y) 


Time 
in  minutes 

Force  L< 
x,   x2 

=vel 

y 

Determi 

0 

5 

5 

5 

1 

27 

5 

5 

4 

1 

50 

4 

5 

4 

1 

55 

1 

5 

4 

1 

56 

*1 

5 

3 

1 

56. 

38 

4 

5 

3 

0 

87 

4 

5 

2 

0 

100 

k 

5 

2 

_ 

Deterministic  Pay-off   -2.0637 
Terminal  State  SI 


1 

-.6231 

1 

-2.1729 

1 

-1.6712 

0 

-1.6373 

0 

-2.0558 

0 

-2.0501 

0 

-2.1846 

_ 

-2.0500 
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Table  VIII.   Comparison  for  x^O)  =  4    x2(0)  =  5    y(0)  =  5 

with  Parameter  Set  *4 

Time    Force  Level  <f>  *  S(tjXl3x,y) 

Deterministic  Stochastic 

0 
0 
0 
0 
0 
0 


Deterministic  Pay-off   -23.2951 
Terminal  State  Sk 

Table  IX.   Comparison  for  x1(0)=2    x2(0)  =  5    y(0)  =  3 

with  Parameter  Set  3 

Time     Force  Level  <J>*  S(t,x1,x2}y) 


in  Minutes 

xi 

X2 

y 

0 

H 

5 

5 

k 

k 

5 

14 

7 

4 

5 

3 

9 

k 

5 

2 

10 

l\ 

H 

2 

13 

l\ 

4 

1 

15. 

51 

H 

H 

0 

1 

-16.9539 

1 

-20.5330 

1 

-23.0807 

0 

-25.3355 

0 

-20.2005 

0 

-20.9796 

_ 

-2*4.0000 

in   Mir 

tutes 

xi 

X2 

y 

Det 

ermi 

0 

2 

5 

3 

1 

13 

2 

5 

2 

1 

18 

1 

5 

2 

1 

31 

1 

5 

1 

1 

35. 

39 

1 

5 

1 

1 

41. 

28 

1 

5 

1 

0 

50 

1 

5 

1 



1 

-8.9286 

1 

-11.1566 

1 

-9.1233 

1 

-10.9620 

0 

-10.7862 

0 

-10.5*4*45 

M 

-10.0000 

Deterministic  Pay-off   -10. 9*471 
Terminal  State  SI 
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Table  X.   Comparison  for  xx(0)  =  4   x2(0)  =  5   y(0)  =  5 

with  Parameter  Set  2 
Time     Force  Level  <J>*  S(t,x  ,x  ,y) 


in  Minutes 

xi 

X2 

y 

Determi: 

0 

3 

5 

4 

1 

9 

3 

5 

3 

1 

13 

2 

5 

3 

1 

20 

2 

5 

2 

1 

34 

2 

5 

1 

1 

36 

• 

1 

5 

1 

1 

41. 

276 

1 

5 

1 

0 

50 

1 

5 

1 

_ 

1 

-8.8300 

1 

-11.21*12 

1 

-8.91^7 

1 

-11.0357 

0 

-12.0296 

0 

-10.8092 

0 

-10.5445 

_ 

-10.0000 

Deterministic  Pay-off   -11. 6727 
Terminal  State  SI 

Table  XI.   Comparison  for  x^O)  =  4    x  (0)  =  5    y(0)  =  5 

with  Parameter  Set  2 
Time    Force  Level  <j)*  S(t,x1Jx2,y) 


in  Minutes 

xi 

X2 

y 

Determi 

0 

4 

5 

5 

0 

7 

4 

5 

4 

0 

13 

4 

5 

3 

0 

19 

•4 

5 

2 

0 

20 

4 

4 

2 

0 

25 

4 

4 

1 

0 

31. 

03 

4 

4 

0 

_ 

1 

-8.4801 

1 

-10.^644 

1 

-11.7339 

0 

-12.6693 

0 

-10.3609 

0 

-11.1032 

_ 

-13 

Deterministic  Pay-off   -11.6476 
Terminal  State  S4 
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Table  XII.   Comparison  for  x  '(0).  »  4    x  (0)  =  .4    y(0)  =  5 

with  Parameter  Set  2 


Time    Force  Level  <f>*  S(t,Xj,x  ,y) 

y   Deterministic  Stochastic 

1 
1 
1 
1 
1 
1 
1 
0 


Deterministic  Pay-off   -9.2835 
Terminal  State  SI 


in  Minutes 

xi 

X2 

y 

0 

4 

4 

5 

7 

4 

4 

4 

10 

3 

4 

4 

16 

3 

1 

3 

23 

2 

4 

3 

26 

2 

4 

2 

39 

2 

1} 

1 

41. 

28 

2 

4 

1 

50 

2 

4 

1 

1 

-6.2034 

1 

-8.8186 

1 

-6.225 

1 

-8.655 

1 

-6.188 

1 

-8.475 

0 

-9.517 

0 

-9.400 

— 

-8.500 
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in  Minutes 

xi 

X2 

y 

•  0 

3 

5 

k 

9 

3 

5 

3 

13 

2 

5 

3 

20 

2 

5 

2 

34 

2 

5 

1 

36 

1 

5 

1 

41. 

28 

1 

5 

1 

50 

1 

5 

1 

1 

-8.1836 

1 

-11.214 

1 

-8.918 

1 

-11.036 

0 

-12.030 

0 

-10.809 

0 

-10.5^5 

_ 

-10.000 

Table  XIII.   Comparison  for  x^O)  =  3   *2(0)  =  5   y(0)  =  4 

with  Parameter  Set  2 


Time    Force  Level  <|>*  S(t,x1}x2,y) 

y   Deterministic  Stochastic 

1 
1 
1 
1 

1 
1 
0 


Deterministic  Pay-off   -11.6727 
Terminal  State  SI 

Table  XIV.   Comparison  for  xx(0)    =  x2(0)  =  y(0)  =  5  with 

Parameter  Set  1 

Time    Force  Level  <j>*  S(t,x1>x2,y) 

y   Deterministic  Stochastic 

1 
1 
0 
0 

Deterministic  Pay-off   -1.5216 
Terminal  State  Si 
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in  Minutes 

xi 

X2 

y 

0 

5 

5 

5 

5.61 

5 

5 

5 

6.38 

5 

5 

5 

26 

5 

5 

k 

50 

5 

5 

4 

1 

-1.509 

0 

-1.446 

0 

-1.435 

0 

-I.665 

, . 

-1.400 

VI.   CONCLUSIONS 

Due  to  the  restrictions  of  the  stochastic  model  to 
small  force  levels  because  of  computer  constraints,  it  is 
difficult  to  reach  any  general  conclusions.   There  are, 
however,  several  Interesting  things  which  can  be  observed 
from  the  data. 

In  the  case  where  a2q>aJp,  the  models  lead  to  target 
selection  rules  where  the  time  fire  is  shifted  differs  by 
as  much  as  seven  minutes.   If  the  deterministic  rule  is  to 
shift  fire  before  X1    is  destroyed,  the  stochastic  rule  is 
to  shift  fire  at  an  earlier  time.   This  can  be  explained  by 
an  interpretation  of  the  model  parameters  similar  to  that 
given  in  Section  III. 

The  return  obtained  by  firing  at  Xl    is  mainly  from 
destroying  his  kill  capability.   More  Y's  survive  for  a 
longer  period  of  time.  '  As  the  battle  progresses,  the 
utility  of  Y  survivors  in  the  future  decreases  until  a 
point  is  reached  where  the  return  from  firing  at  X  drops 
off  rapidly.   Conversely,  the  return  from  firing  at  X   is 
chiefly  from  destroying  his  value.   As  the  battle  progresses 
the  return  from  firing  at  X,  eventually  exceeds  that  of 
firing  at  X  .   In  the  deterministic  model,  this  crossover 
point  is  known  precisely.   On  the  other  hand,  this  cross- 
over point  is  not  precisely  known  in  the  stochastic  model. 
The  use  of  expected  values  tends  to  force  an  earlier  shift 
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to  keep  away  from  the  area  where  the  X   return  decreases 
rapidly.   In  other  words,  some  of  the  future  value  of  Y 
survivors  is  sacrificed  to  ensure  that  the  higher  return 
from  firing  at  X2  is  not  missed.   In  the  case  where 
a1p>a2q,  this  does  not  occur  since  it  is  always  to  Y's 
advantage  to  fire  at  Xl5  both  in  terms  of  value  destroyed 
and  future  survivors. 

The  large  jump  in  the  optimal  expected  value  function 
as  casualties  occur  is  caused  by  the  integer  constraints 
on  the  stochastic  model.   This  does  not  happen  in  the 
deterministic  model  since  force  levels  are  calculated  as 
continuous  functions.   As  t  ■*  T,  the  optimal  expected  value 
function  approaches  the  deterministic  pay-off. 

There  are  a  number  of  areas  which  need  additional  work. 
Most  obvious  is  the  need  for  better  numerical  techniques 
for  the  stochastic  model.   One  alternative  is  to  reprogram 
the  model  to  use  tape  storage  rather  than  core.   In  this 
way,  a  series  of  small  computer  runs  could  be  used  to  build 
up  a  file  of  values  for  smaller  force  levels  which  could 
then  be  accessed  to  compute  still  higher  force  levels.   In 
this  case,  an  investigation  of  the  truncation  errors  would 
have  to  be  made  to  determine  the  degree  of  accuracy  in  the 
figures  for  higher  force  levels. 

Another  alternative  is  to  consider  a  diffusion  approxi- 
mation to  the  stochastic  process.   Some  work  in  this  area 
has  been  done  by  Taylor  [10],  leading  to  a  parabolic  partial 
differential  equation  which  is  not  easily  solved 
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analytically.   It  can  be  solved  numerically.   This  approach 
appears  promising  since  it  would  not  be  necessary  to  solve 
for  all  combinations  of  force  levels  below  the  desired  ones. 

This  approach  to  the  stochastic  model  yields  no  informa- 
tion about  the  time  history  of  the  average  force  levels 
or  the  probability  of  winning.   Clark  [16]  was  able  to 
obtain  expressions  for  the  average  force  levels  in  the 
linear  law  case.   If  a  similar  solution  for  the  square  law 
case  could  be  obtained,  more  meaningful  comparisons  could 
be  made . 

There  are  also  a  number  of  extensions  to  the  determin- 
istic model.   The  zero-one  nature  of  the  solution  is  hardly 
realistic.   It  is  a  result  of  the  square  law  attrition 
mechanism  and  the  constant  attrition  coefficients.   One 
area  of  investigation  would  be  to  make  the  attrition  coef- 
ficients a  function  of  the  fire  being  received.   Other 
areas  of  extension  are  the  two-on-two,  one-on-m,  and  m-on-n 
battles  with  various  types  of  attrition  mechanisms  and 
coefficients . 

In  summary,  the  numerical  results,  while  inconclusive, 
do  indicate  a  significant  difference  in  results  between 
the  stochastic  and  deterministic  modelling  approaches . 
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APPENDIX  A. 
Domains  of  Controllability 


Case  I.   axp>a2q 


fx  (T)  >  0 


SI   i 


x  (T)  >  0 


4>*(t)  =  1 


o  <  t  <  T 


y  (T)  >  o 

Requires  T  <  TS1  where 


TSlH 


1 


/albl 


In  {Y}    if  a^y^O)  >  s 


In  - 


/albi 


/a1b1 


b2x2(0) 


if  aibiy2(0)  =  s 


In  {-Y}    if  s2-  (b2x2(0))2  _  a1b1y2(0)<! 


tanh     < 


Vajbj  y(0) 


ZaTb" 


if  aibiy?-(0)  <  s2  -  (b2x2(0)) 


Y  = 


/a^bjT^O)  -  s2  +  (b2x2(0))2  -  b2x2(0) 


^ajbj  y(0)  -  s 


'xj(TSl)  =  0 


S2  \x2(T)    >  0 


y(t)  >>o 


4>*(t) 


'l    0  1  t  <    TS1 

TS1  <  t  1  T 
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Requires  a1b1y2(0)  >  s2  --  (t>2x2(0)) 


TS1  <  T 


S3 


S5 


rxx(T)    0 


x2(T)  =  0 


ky(T)  >  0 


fXjCTl)  >  0 


No  extremals  lead  to  this  state 


sh 


x2(Tl)  >  0      <j>*(t)  =  1 

y(Tl)  =  0 

Requires  a1b1y2(0)  <  S   -  (b2x2(0)) 

Tl  <  T 


0  <  t  <  Tl 


fx^TSl)  =  0 


x  (Tl)  >>0 

2 


(1  0  <  t  £  TS1 

0     TS1  <  t  <  Tl 


y(Ti)  =  o 

Requires  S2  -  (b2x2(0))2  _  aibjy2(0)  <  s 
+  (R-l)(b2x2(0))2 
Tl  <  T 
'xjCTl)  >  0 
S6   ■ x2(Tl)  =  0 

ky(Tl)  =  o 

'x^TSl)  =  0 


No  extremals  lead  to  this  state 


S7   -|x2(Tl)  =  0 


<(>*(t)  =  • 


(l  0  <  t  <  TS1 

0     TS1  <  t  <  Tl 


;y(Tl)  >  0 
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Requires  a  by  (0)  >  s2  +  (R-l)(b  x  (0)) 

Tl  .  T 


Xj(Tl)  =  0 
'  x2(Tl)  =  0 

.y(Ti)  >  o 


No  extremals  lead  to  this  state 


Case  II.  axp  <  a2q 
x1{1)    >    0 


SI 


fO         0    <t<    T      if   TS1    > 


x„(T)    >    0  <j)*(t)    -    -1         0    <    t    <    T    -   TS1 


ky(T)    >    0 


where   tSI   = 


0        T  -  tSI  <  t  5  T  if  tSI  <  T 


/a2b2 


In 


Z    + 


/vT 


-  i 


1    +   Y 


Requires    T  <    TSI   where 


TSlH 


/albl    y(0) 
tanh  }■    if   r1  b1  y(0) 


/a2b2 


>/R 


<    R{s2    -fb1  x2  (0))  2} 
- —  In   {y}       if   R{  s2    -    (b^CO))2}!   ax  bx  y2  (  0  )<Rs< 


v/a2  b2 


1        .  s 
In 


/a2b2 


b2x2  (0) 


if   Si1b1  y2  (0)    =   Rs: 


- —  In  {-y>       if  Rs2    <    axbj  y2  (0) 


/a2  b2 
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Y  = 


jXjCO)  /R  -  /a^y^O)  -  R{s2  -  (bjX^C))2} 
s/R  -  Za7b7  y(0) 


S2 


fl     0  <  t  <  TS2 
0     TS2  <  t  <  Tl 


'x  (TS2)  >  0 
x2(T)  >  0       (J,*(t)  = 

y(T)  >  o 

Requires  a  b  y2(0)  >  s2  -  (b2x2(0))   and 

tSI  +  TS2  <  T  where 

In  (y)     if  Ei1b1y2(0)    >    s2 


TS2  = 


r   i 

1 

■^ 

/aibi 

1 

/a1b1 

In 


b2x2(0) 


a1bly2(0)  =  s2 


In  {-y>    if  a1b1y2(0)  <  s 


Y 


/a^y^O)  -  s2  +  (b2x2(0))2  -  b2x2(0) 
/a7F7  y(0)  -  s 


S3 


fXjCTSS)  >  0 


x  (T)  =  0       cj)*(t)  = 


'1     0  <    t    <    TS3 
0     TS3  <  t  <   T 


y(T)  >  o 


Requires  TSI  <  T  and  T  -  TSI  <  TS3  where  TS3  is  a 
root  of 


0 


i(l   +  —  )[s    cosh    (y.)    -    /a7b7y(0)    sinh    (y.)] 
2  /R  ill 
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+  Id 


/R 


)[s    cosh    (y2)    -    Za7b7  y(0)    sinh    (y2)] 


-   s    cosh    (/a^  TS3) 


+    /a1b1    y(0)    sinh    (/a7b7TS3)    +   b2x2(0) 


Y,    =    /a7B7   (T   -   TS3)    +   Sa^  TS3 


Y2    =    /a2b2    (T  -   TS3)    -   /a^    TS3 
'Xj (Tl)    >    0 


Sk      -!x2(Tl)    >    0 


^y(Ti)  =  o 


TS4     = 


/a2b2 


In 


Z    +  /  Z      -    1 


(a.)    When   xS4  >    Tl  4>*(t)    =   0 

Requires 


0    <    t    <    Tl 


axbiy2(0)    <    Rs2    |    1    -  4  } 
aibiy2(0)    <    R   {s2    -    (blXl(0))2} 
aibiy2(0)    <    s2    +    (R   -    l)(b2x2(0)) 


Tl   i   T 

(b. )    When  xS4   <    Tl 

(1.)    When   R    -   /R(R    -    1)    <    6    <    1 

fl         0    <    t    <    Tl    -   TS4 
0        Tl    -   ts4    <    t   <    Tl 


4>*(t)   =  J 
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Requires 

a^y^O)  >  s2  +  A(b2x2(0)) 

aibiy2(0)  <  s2  +  B(b2x2(0)) 

a^y'tO)  >  Rs2  {  1  -A.} 

Tl  <  T 


(2. )  When  6  =  R 
(1 


cf>*(t)  = 
Requires 


0 


/R(R-l) 
0  <  t  <  TS'4 
TS4  <  t  <  Tl 


a^y^O)  =  s2 

b2x2(0)  >  blXl(0){/^I-  1 


Tl  <  T 


where  TS^  is  any  value  less  than  TS2 


S5 


fx  (TS2)  =  0 
l 


x2(Tl)  >  0 


y(Ti)  =  o 


(j)*(t)  = 


fl    0  <  t  <  TS6 

0    TS6  <  t  <  Tl 


Requires 

alb1y2(0)  >  s2+B(b2X2(0))2 
aibiy2(0)  <  s2  +  (R-l)  (b2x2(0)) 
Tl    T 
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S6 


'x^Tl)  >  0 


x2(Tl)  =  0 


.y(Ti)  =  o 


4>*(t)  = 


1     0  <    t  <  TS6 
0     TS6  <  t    Tl 


Requires 

a^y2  <s2  +  A(b2x2(0))2 
aibiy2(0)  >  R{s2  -  (blXl(0))2} 
aibiy2(0)  <  s2  +  (R-l) (b2x2(0)) 
Tl  <    T 


where 


TS6  = 


/aib7 


/i7F7 


In  (y)    if  axbjy2(0)  >  s 


ln  {  yff§5y}lf  a1b1y2(0)  =  a 


- —  In  (-y)   if  a1b1y2(0)  <  s 


•Vi 


where 

y(TS6) 


/y2(TS6)  -  y2(0)  + 


S2 


Y 


and 


aibi 


y(o)  - 


/albl 


X(o) 


y(TS6)  =  /  ~ i(2R-l)b2x2(0) 


+  2/s2  +  R(R-l)[b2x2(0)J^  -  a.lb1y{0)2} 
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'xjCTl)    =    0 

'l 

S7     ■ 

X2(T1)    =    0 

Ly(Ti)   >  o 

4>*(t)   =  « 

0 

0  <  t  <  TS2 
TS2  <  t  1  Tl 


Requires 

a^y'CO)  >  s2  +  (R-l)  (b2x2(0)) 
Tl  1  T 


S8   ■ 


Xj(Tl)  =  0 
x2(Tl)  =  0 

ky(Ti)  >  o 


No  extremals  lead  to  this  state 
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